hai St 
ff j ; 
id at ISSN: 0975 3893 


IJBHTM (2011), 3(1):1-5 
A Structured Framework to Establish and Evaluate Multidimensional 
Outcome Measures in Drug Development: A Conceptual Proposal 


Hui Xiang! and Yi Zhu’ 


‘Arizona School of Health Sciences, A.T. Still University, Mesa, Arizona 85206, U.S.A. 
*Chengdu No. 1 People’s Hospital, Chengdu, Sichuan, CHINA 610041 


hxiang @atsu.edu 


ABSTRACT: 

This paper proposes a structured framework to establish multidimensional outcome measures in drug development. In addition to 
clinical results, the outcome measure also take into consideration of patient health related quality of life (HRQOL) at different 
functional levels and acceptance from stakeholders such as clinicians, caregivers, patients, manufacturers, and payers. The outcome 
measures are evidence-based with clearly identified critical quality attributes (CQAs). The goal attainment of multidimensional 
outcome measures can be visually assessed using graphic illustrations, such as Radar chart. The master outcome measures (MOM) 
of a drug development program are defined by clinical results, patient HRQOL, and acceptance, which are further defined by sets of 
sub-level outcome measures. The outcome measures are validated using the following nine criteria: model development, 
appropriateness, reliability, validity, responsiveness, precision, interpretability, acceptability, and feasibility. Risk assessment and 
multivariable methods (design space) are used to establish the design space between outcome CQAs and critical/key factors using 
both forward (rationale-oriented) and backward (results-oriented) approaches. Outcome measures should be actively managed and 
updated throughout the drug product lifecycle. 


Keywords: quality by testing (QbT), quality by design (QbD), outcome-driven approach, multidimensional outcome measures, 
Radar chart, health related quality of life (HRQOL), critical quality attribute (CQA), outcome validation, design space, multivariable 
methods, risk assessment, drug development, product lifecycle 


INTRODUCTION 

The traditional approach in drug development used to 
focus heavily on testing of drug product (Figure 1), 
which is sometimes referred to as the quality by testing 
(QbT) approach. The high variation and failure rate 
due to lack of control in the manufacturing process 
made people aware that “quality cannot be tested into 
products, it has to be built in by design” [1]. However, 
the drug industry lags behind other industries such as 
semiconductor industry in employing the quality by 
design (QbD) concept. The past decade saw the 
pharmaceutical industry started to embrace QbD 
methodologies in drug development process [2,3], 
which greatly enhanced the control in drug production 
process and dramatically reduced the different types of 
risks in the process. However, the current QbD 
approach focuses mainly on the CQAs of a drug 
product and the critical parameters of drug 
manufacturing process (Figure 1), without a systematic 
approach on defining outcome measures of drug 
development. Without a well-defined outcome, it is 
difficult to assess the value and quality potential of a 
drug program [4]. For example, many companies are 
still struggling on how to establish the CQAs of a drug 
product [5]. In the past, the best possible clinical 
results on medical conditions are automatically the 
target the development is shooting at. But do clinical 
results tell the whole story? As early as 2001, the PRO 
Harmonization Group task force report urged to 
incorporate the patient’s perspective into drug 
development and communication, in addition to 
clinical results [6]. Until today, though the need for 
extensive outcome research beyond clinical results was 


well recognized [7], there is no consensus on how to 
establish the outcome measures in drug development. 


This paper tried to explore the multidimensional 
outcome measures for drug development, with the 
objective to propose a structured framework to 
establish outcome measures in order to have balanced 
assessment in drug development. In addition to 
clinical results, we need to add other dimensions, such 
as patient HRQOL at different functional levels, and 
acceptance from different stakeholders such as 
clinicians, patients, caregivers, manufacturers, and 
payers. 


OUTCOME-DRIVEN APPROACH IN DRUG 
DEVELOPMENT 

In contrast to the traditional QbT and current QbD 
approaches, the outcome-driven approach targets the 
best possible results in not only clinical results, but 
also patient HRQOL and acceptance from different 
stakeholders, as illustrated in Figures 1 and 2. The 
clinical results will consider the — efficacy, 
pharmacokinetics/pharmacodynamics (PK/PD), safety, 
immunogenicity of a drug product [8]. The patient 
HRQOL will take into consideration of factors 
affecting patient quality of life at different levels, 
namely cell/tissue level, organ level, individual level, 
and social (and environment) level. This covers a 
variety of patient feedback, such as emotional/mental 
health, social function, and general well being, on their 
experience and satisfaction when using a drug product 
in their daily life at different time frames. The 
acceptance will consider acceptance and preference 
from different stakeholders, namely patients, 
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caregivers, clinicians, manufacturers, and payers. 
Thus, the outcome-driven approach takes a systematic 
look at overall outcomes in a drug development 
program from different perspectives and stakeholders. 
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Figure 1. The Multidimensional Outcome Measures in Drug Development 
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Figure 2. Different Approached in Drug Development. Quantitative multi-variable characterization is 


schematically illustrated by oval circles. 


GRAPHIC ILLUSTRATION OF 

MULTIDIMENSIONAL OUTCOME MEASURES 
Multidimensional outcome measures can be visualized 
by Radar chart, which is schematically illustrated in 
Figure 3. Each axis represents one dimension of an 


outcome. The length of an axis represents the total 
score, which is lined up with other total scores to form 
the total area of a set of outcome measures. The length 
of an axis may vary, depending on the pre-determined 
criticality of an outcome, though Figure 2 shows all the 
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axes are equivalent. The actual score of each outcome 
is also lined up to form the actual area of this set of 
outcome measures. The goal attainment can be 
assessed graphically by comparing the actual area 
(colored) to the total area, which will help to identify 
strength and weakness. 
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Figure 2. Schematic Illustration using Radar Chart to Evaluate Goal Attainment in Drug Development 


MULTIDIMENSION OUTCOME MEASURES 
AT DIFFERENT LEVELS 

The master outcome measures (MOM) define the 
overall goal of a drug development program, which is 
tentatively defined by three factors: clinical results, 
patient HRQOL, and acceptance. Each of these factors 
is the 2nd level of outcome measures, which is defined 
by a set of 3rd level of outcome measures, and so on 
(Figure 1). Alternatively, the fish bone diagram can be 
used to illustrate the relationship between the different 
levels of multidimensional outcome measures. 


DEFINING OUTCOME MEASURES AND 
THEIR DETERMINANTS 

Outcome measures are evidence-based with clearly 
identified CQAs using a combination of forward 
(rationale design) and backward (result-oriented) 
approaches (Figure 4). The initial outcome measures 
are established by the forward approach using a 
combination of methodologies such as_ rationale 
design, prior knowledge, and risk assessment, which, 
as more data become available in drug development, 
will be re-evaluated and updated by the backward 
approach using a combination of methodologies such 
as design space (with clearly defined CQAs), trending 
and distribution analysis, qualification and validation 


studies, etc. The link between outcome measures and 
the determinants should be established and well 
characterized by a variety of quantitative multi- 
variable methods to ensure it is well understood and 
well controlled with minimal variation (Table 1). 


Risk assessment should be conducted throughout the 
product lifecycle (Figure 4). The commonly used risk 
assessment tools, such as failure mode and effect 
analysis (FMEA) and hazard analysis and critical 
control point (HACCP), are not appropriate to assess 
risks in early development stages as these tools may 
take off risks too early due to low scores on occurrence 
and/or detection based on assumption or limited data 
[5]. Instead, there are several proposed tools for early 
CQA assessment. For example, the one proposed by 
Genetech is to assess the criticality of each quality 
attribute by impact and uncertainty on drug safety and 
efficacy, while the other one proposed by MedImmune 
assesses the criticality by severity and likelihood 
instead [8]. As more data become available, more 
comprehensive risk assessment programs’ with 
quantitative risk calculation and sufficient control 
strategy should be established to assess the real risks in 
every aspect of product production and delivery. 
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Figure 4. Approaches and Methodologies Employed in Early and Late Stage Drug Development 


Table 1. Quantitative Multi-variable Characterization in Drug Development 


-Effect (Physical, Biological) 


-Component (Complex, Simple, Source) 


Contact Surface “lmpanuss 
Characterization eacteb les 
-Extractables 
-Effect (Physical, Biological) 
-Design 
; -Installation 
Equipment : 
nae -Operation 
Characterization 
-Performance 
-Calibration 
-Design (Work Segregation, Air Flow and Quality, Consumables, 
Facility Wastes, HVAC, etc.) 
Characterization -Operation (Clean, Maintenance, Upgrade, Monitor) 


-Performance (Environment, Breach, Safety, Security) 


controlled 
(parameters and 
ranges) process to 
produce expected 
outcome with 
minimal variation. 


Task Description Goal Methods 
-Criticality 
Outcome -Variability 
Characterization -Controllability 
-Acceptability 
-Structure (primary structure, secondary and high order structure) 
Product -Pre-formulation 
Characterization -Stability 
-Function (activity, potency, etc.) 
-Microbiology (Cell Bank, Seed Culture) -Comprehensive risk 
Process -Fermentation (Culture, Harvest) The goal of management program 
Characterization -Purification (Chromatography, Precipitation, Filtration) characterization is to (systematic risk 
-Formulation establish well- assessment, control 
Raw Material -Component (Complex, Simple, Source) understood (CQAs strategy, comprehensive 
cuits -Impurities and effects) and well- | review, etc.) 
Characterization 


-Design space (DOE) 
-Robustness studies 
-Challenging studies 
-Trending and 
Distribution analysis 
-Process analytical 
technology (PAT) 


VALIDATING OUTCOME MEASURES 

Outcome measures should be carefully developed so 
that they serve the intended purpose of setting 
appropriate goals in drug development. A systematic 
process should be followed to take into consideration 
of different factors in across the different perspectives 
from drug development to intended use, which involve 
item generation, testing, item reduction, and scale 
development. The outcome measures need to be 
validated to ensure it is appropriate and serves their 
intended purposes. As proposed by [9-11], the 
following nine criteria can be used in outcome measure 


validation: scale development, appropriateness, 
reliability, validity, | responsiveness, _ precision, 
interpretability, acceptability, and feasibility (Table 2). 
While general concept can be followed in developing 
outcome measures, they cannot be used 
indiscriminately without considering the specific 
situations in individual cases. Also, there is no scale 
that ideally satisfies all the criteria, hence, drug 
developers need to consider the relative advantage and 
limitations of various outcome measures and develop 
the measures best suit their needs. 
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Table 2. Evaluation Criteria for Outcome Measures 
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Criteria Definition Establishing Methods 
Scale Effect size in an outcome Systematic process (follow the steps of item generation, initial item 
measure reduction, testing, final item reduction, establishing scale measurement 
properties) 
Appropriateness Comparing an outcome measure | Rationale, review 


to its intended purpose 


Reliability (Internal consistency 
and reproducibility) 


Compares random error to the 
true effect 


Internal consistency and reproducibility 


Interclass correlation coefficient (ICC), Cronbach’s alpha 0.7 — 0.9, 
Pearson r correlation coefficient 


Validity (criterion, face/content, 
construct) 


Ability of an instrument to 
measure intended outcome 


Comparison to “gold standard”, concurrent and predictive, outcome, 
construct 


Responsiveness Ability to detect change over Change score, effect size, standardized response mean (SRM), modified 

time when changes happen SRM, relative efficiency, sensitivity and specificity, ceiling and floor 
effects 

Precision Consistency among individual binary, likert, VAS 
measurements 

Interpretability Ability to interpret the meaning Rationale, design, feedback 
of the scores 

Acceptability Ease to provide feedback Rationale, design, feedback 

Feasibility Ease to administer Rationale, design, feedback 


Note. Prepared based on references [9-11]. The table title was revised on February 20, 2012 from the originally published manuscript. 


CONCLUSION 

This paper proposes an outcome-driven approach in 
drug development and a structured framework to 
establish multidimensional outcome measures at 
different levels. In addition to clinical results, the 
outcome measure also take into consideration of 
patient HRQOL at different functional levels and 
acceptance from stakeholders such as clinicians, 
caregivers, patients, manufacturers, and payers. The 
goal attainment of the outcome measures can be 
graphically illustrated by Radar chart. The outcome 
measures are defined using a combination of forward 
and backward approach at different stages in drug 
development. The outcome measures need to be 
validated to ensure they are appropriate and serve the 
intended purposes. This is merely a conceptual 
proposal, drug developers need to consider the relative 
advantage and limitations of various outcome 
measures and develop the measures best suit their 
needs. 
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